feat: Sync from noir #11294

AztecBot · 2025-01-17T08:06:04Z

Automated pull of development from the noir programming language, a dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
fix: don't always use an exclusive lock in nargo check (noir-lang/noir#7120)
feat(ssa): Pass to preprocess functions (noir-lang/noir#7072)
chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105)
fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100)
feat: Parser and formatter support for enums (noir-lang/noir#7110)
feat(brillig): SSA globals code gen (noir-lang/noir#7021)
feat: loop keyword in runtime and comptime code (noir-lang/noir#7096)
chore: Add benchmarking dashboard (noir-lang/noir#7068)
feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088)
feat(ssa): Add flag to DIE pass to be able to keep store instructions (noir-lang/noir#7106)
chore: Cookbook Onboard integration (noir-lang/noir#7044)
chore: lock to ubuntu 22.04 (noir-lang/noir#7098)
fix: Remove unused brillig functions (noir-lang/noir#7102)
chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
END_COMMIT_OVERRIDE

… in global space (noir-lang/noir#7095)

…oir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

…oir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

…oir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

…oir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

…oir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

…oir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

* master: (442 commits) feat(p2p): request response node sampling (#11330) chore: Trace propagation from json rpc client to server (#11325) feat(p2p): send goodbye messages on disconnecting to peers (#10920) fix(bootstrap): include crates in noir projects hashes (#11344) chore: Add OTEL_EXCLUDE_METRICS (#11317) chore: Test starting multiple anvils allocates distinct ports (#11314) fix: move eslint in circuits.js to dev deps (#11340) fix: update devbox (#11339) chore(p2p): reorganise reqresp handlers (#11327) fix(readme): remove stale link (#11333) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg chore: bump CRS and constants (#11306) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg chore: redo typo PR by nnsW3 (#11322) ...

…oir-lang/noir#7120) feat(ssa): Pass to preprocess functions (noir-lang/noir#7072) chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105) fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

…oir#7120) feat(ssa): Pass to preprocess functions (noir-lang/noir#7072) chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105) fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)

aakoshh · 2025-01-20T22:47:30Z

Managed to reproduce this failure on the mainframe as follows:

Build the system:

./bootstrap.sh clean
git fetch origin
git checkout --track origin/sync-noir
./bootstrap.sh full
#npm install --global corepack --force
#yarn set version 4.5.2; yarn install

It changed since I last tried and takes a long time, but succeeded.

Then we need to build images and run the test that ci runs; luckily there is bootstrap entry point for it, which also takes care of building the necessary docker images:

./bootstrap.sh test-e2e -- e2e_fees_failures

Alternatively the images can be built as follows:

./bootstrap.sh image-e2e

This shows the same failure as on CI. After this when we change something in the Noir code under noir/noir-repo, we can run ./bootstrap.sh again to rebuild nargo and all the contract and protocol circuits, which unfortunately takes a really long time, without indicating what's happening (haven't found a way to enable verbose logging yet); it's not yet clear what is the minimum that needs to be rebuilt. Altogether it was about 15 minutes for another test run to start (noir-projects/joblog.txt contains timings for the 3 contract directories).

Now, when we run ./bootstrap.sh image-e2e again it says the image already exists, because it uses the git hash to determine the tag, which has not changed (we're using the image pulled from CI). To trigger a change we need to make a git commit, and then rebuild the images locally.

Unfortunately this doesn't work, complaining that the commit wasn't pushed:

Earthly error

================================== ❌ FAILURE ===================================

  +bootstrap-noir-bb *failed* | Repeating the failure error...
  +bootstrap-noir-bb *failed* | --> RUN --raw-output bash -c "$bootstrap_noir_bb"
  +bootstrap-noir-bb *failed* | The commit was not pushed, run aborted.
  +bootstrap-noir-bb *failed* | ERROR Earthfile:32:2
  +bootstrap-noir-bb *failed* |       The command
  +bootstrap-noir-bb *failed* |           RUN --raw-output bash -c "$bootstrap_noir_bb"
  +bootstrap-noir-bb *failed* |       did not complete successfully. Exit code 1
    +bootstrap-aztec | apply build +bootstrap-with-verifier: earthfile2llb for +bootstrap-with-verifier: Earthfile:68:2 copy artifact: apply build +rollup-verifier-contract-with-cache: earthfile2llb for +rollup-verifier-contract-with-cache: Earthfile:268:2 failed to expand LET value "rollup-verifier-contract-$(./noir-projects/bootstrap.sh hash).tar.gz": failed to process "rollup-verifier-contract-$(./noir-projects/bootstrap.sh hash).tar.gz": non constant build arg read request: process "/bin/sh -c CARGO_HOME=/opt/rust/cargo EARTHLY_GIT_HASH=ebb07743425a702b0213e4ccd389aa617a9cdf1a GITHUB_LOG=1 PATH=/opt/rust/cargo/bin:/opt/foundry/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin RUSTUP_HOME=/opt/rust/rustup bootstrap_noir_bb='rm -rf $(ls -A) &&\n    git init >/dev/null 2>&1 &&\n    git remote add origin https://github.com/aztecprotocol/aztec-packages >/dev/null 2>&1 &&\n    (git fetch --depth 1 origin $EARTHLY_GIT_HASH >/dev/null 2>&1 || (echo \"The commit was not pushed, run aborted.\" && exit 1)) &&\n    git reset --hard FETCH_HEAD >/dev/null 2>&1 &&\n    ./build-images/adhoc-installs.sh &&\n    DENOISE=1 CI=1 ./noir/bootstrap.sh fast &&\n    DENOISE=1 CI=1 ./barretenberg/bootstrap.sh fast &&\n    mv $(ls -A) /usr/src' AWS_ACCESS_KEY_ID=\"$(cat /run/secrets/AWS_ACCESS_KEY_ID)\" AWS_SECRET_ACCESS_KEY=\"$(cat /run/secrets/AWS_SECRET_ACCESS_KEY)\" /usr/bin/earth_debugger /bin/sh -c 'bash -c \"$bootstrap_noir_bb\"'" did not complete successfully: exit code: 1
    +bootstrap-aztec | in		github.com/AztecProtocol/aztec-packages:sync-noir+rollup-verifier-contract-with-cache
    +bootstrap-aztec | in		github.com/AztecProtocol/aztec-packages:sync-noir+bootstrap-with-verifier

Help: To debug your build, you can use the --interactive (-i) flag to drop into a shell of the failing RUN step: "earthly -i --secret AWS_ACCESS_KEY_ID=XXXXX --secret AWS_SECRET_ACCESS_KEY=XXXXX --secret S3_BUILD_CACHE_UPLOAD=XXXXX --secret S3_BUILD_CACHE_DOWNLOAD=XXXXX --secret AZTEC_BOT_COMMENTER_GITHUB_TOKEN=XXXXX --artifact +bootstrap-aztec/usr/src /tmp/tmp.LWZns6APgS/usr/src"

🛰️ Reuse cache between CI runs with Earthly Satellites! 2-20X faster than without cache. Generous free tier https://cloud.earthly.dev

Trying again after creating a branch for my investigations:

git add noir
git commit -m "DEBUG"
git checkout -b af/sync-noir-debug
git push --set-upstream origin af/sync-noir-debug

This time it does the build, but it looks like it's running ./bootstrap.sh full inside the Docker build process again 😣

Okay so either I figure out a different way of running the test script directly against a running container, or use this workflow, but it's less likely that I can limit the build if it has to rebuild an e2e image from scratch. FWIW according to e2e_test_config.yml I have to run e2e_fees/failures.test.ts.

EDIT: It's been building the image for over 30 minutes now, at the moment creating the VK for the second protocol contract, which seem to take ages. I used to be able to run these; maybe at the time it ran natively using more CPUs. I'll check it in the morning.

EDIT2: After about 10 more minutes the test ran. I can confirm that disabling the Ssa::preprocess_fns pass the tests do not fail.

AztecBot and others added 13 commits January 17, 2025 08:05

[1 changes] chore(ssa): Use correct prefix when printing array values…

5a889cb

… in global space (noir-lang/noir#7095)

chore: apply sync fixes

9f63bed

chore: apply sync fixes

a72b1b8

chore: apply sync fixes

470f921

chore: apply sync fixes

1ae9dc8

Merge branch 'master' into sync-noir

cde04e6

TomAFrench approved these changes Jan 20, 2025

View reviewed changes

AztecBot added 3 commits January 20, 2025 14:55

chore: apply sync fixes

f21b821

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Sync from noir #11294

feat: Sync from noir #11294

AztecBot commented Jan 17, 2025 •

edited

Loading

aakoshh commented Jan 20, 2025 •

edited

Loading

feat: Sync from noir #11294

Are you sure you want to change the base?

feat: Sync from noir #11294

Conversation

AztecBot commented Jan 17, 2025 • edited Loading

aakoshh commented Jan 20, 2025 • edited Loading

AztecBot commented Jan 17, 2025 •

edited

Loading

aakoshh commented Jan 20, 2025 •

edited

Loading